• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The invention relates to a method for assisting the maintenance of an aircraft, implemented in a system (100) comprising: - display selection means (130); and - a portable apparatus (120) with a camera (121) and augmented reality display means (122). The method comprises the following steps: the camera (121) acquires images (Im) of a piece of equipment from the aircraft, and sends them to the display selection means (130); the display selection means (130) identify the equipment present on these images, and determine its identifier (Id), said useful identifier; from the useful identifier (Id), the display selection means (130) sends to the augmented reality display means (122) maintenance assistance data; in response, the augmented reality display means (122) display, in fact augmented, images (C) corresponding to said data. The invention also relates to a device for implementing such a method.
    公开号:FR3023948A1
    申请号:FR1457015
    申请日:2014-07-21
    公开日:2016-01-22
    发明作者:Siegfried Soldani
    申请人:Airbus Operations SAS；
    IPC主号:
    专利说明:

    [0001] METHOD FOR AIDING THE MAINTENANCE OF AN AIRCRAFT BY INCREASED REALITY TECHNICAL FIELD The present invention relates to the field of assistance to the maintenance of an aircraft. STATE OF THE PRIOR ART When a failure occurs on an aircraft, the maintenance center of the airline is informed thereof, and provides one or more maintenance operation (s) to be performed on the aircraft. The airline maintenance center may also provide one or more preventive maintenance operation (s). Each maintenance operation comprises for example a replacement, manipulation and / or repair of at least one aircraft equipment. Such equipment is, for example, a brake of a landing gear, a wheel of a landing gear, a control system of the wheel, etc.
    [0002] A list of these maintenance operations is provided to a technician to carry out these operations. The aircraft is immobilized for the duration of the intervention of the technician. The list of operations is accompanied by details relating to each operation, to accompany the technician in his task. These precisions comprise a list of elementary steps relating to each operation of the list. This ensures in particular that the elementary steps associated with a maintenance operation are implemented in an order to optimize the duration of immobilization of the aircraft.
    [0003] An object of the present invention is to provide a method and a device for aiding the maintenance of an aircraft, further reducing this immobilization time.
    [0004] SUMMARY OF THE INVENTION This object is achieved with a method of assisting the maintenance of an aircraft, implemented in a system comprising: - display selection means; and a portable apparatus comprising a camera and augmented reality display means. The method according to the invention comprises the following steps: the camera acquires images of a device of the aircraft, and sends these images to the display selection means; the display selection means identify the equipment present on said images, and provide an identifier of the identified equipment, said useful identifier; from this useful identifier, the display selection means selects maintenance assistance data relating to the identified equipment, and sends these data to the display means actually augmented; in response, the augmented reality display means, in fact augmented, display images corresponding to the maintenance assistance data. Thus, a user equipped with the portable device according to the invention observes a view of a device, on which are superimposed the images corresponding to the maintenance assistance data. The display implemented by the augmented reality display means allows the user equipped with the portable device according to the invention to quickly identify a device on which he must perform at least one step of an operation. of maintenance. This reduces the search time of this equipment. In addition, this display allows this user to easily view information relating to the implementation of the maintenance operation. This reduces a duration to access such information. Thus, the maintenance assistance method according to the invention makes it possible to reduce a period of immobilization of an aircraft on which a maintenance operation is carried out. Preferably, the display selection means implement the following steps: receiving a list of at least one identifier of interest, each identifier of interest corresponding to a device of interest; - comparison between the useful identifier and at least one identifier of interest of the list; and selecting and sending said maintenance aid data only when the useful identifier corresponds to an identifier of interest in the list. The display selection means advantageously implement the following steps: receiving a request relating to at least one maintenance operation to be performed; extraction, from this request, of the list of at least one identifier of interest, an identifier of interest corresponding to a device of interest, and the equipment of interest forming a device associated with one of the at least one maintenance operation to be performed. The display selection means can implement an extraction, for each identifier of interest from the list, and from the request relating to at least one maintenance operation to be performed, a position of the equipment of corresponding interest. The display selection means may implement an extraction, for each identifier of interest from the list, and from the request relating to at least one maintenance operation to be performed, corresponding maintenance assistance data. . Preferably, the portable apparatus provides the display selection means with a position of the portable apparatus. The portable device advantageously implements the following steps: determining a raw position of the portable device, relative to a reference point; comparison between this raw position and a modeling of the aircraft; determining the position of the portable apparatus relative to said aircraft. To identify the equipment present on said images, the display selection means may select a list of equipment located near the portable device, from the position of the portable device and the positions of the equipment of the device. 'aircraft. The display selection means may implement a comparison between the position of the portable device and a position of a device of interest, the equipment of interest corresponding to an identifier of interest of a list of devices. identifiers of interest such that the display selection means send maintenance assistance data only when the useful identifier corresponds to one of the identifiers of interest in the list. Preferably, the method according to the invention then comprises the following steps: the display selection means selects and sends to the display means actually augmented, guidance data to a device of interest; and the augmented reality display means displays, in augmented reality, images corresponding to these guidance data. The method according to the invention advantageously comprises the following steps: - the display selection means selects and sends to the display means actually augmented, alert data relating to the proximity of a device of interest, when the position of the portable device corresponds to the position of a device of interest; and the augmented reality display means, in fact augmented, display images corresponding to these alert data.
    [0005] The method according to the invention may comprise a step of formulating, by a human operator and using a human-machine interface, the request relating to at least one maintenance operation to be performed. The method according to the invention may comprise a step of sending, by a maintenance center, the request relating to at least one maintenance operation to be performed. The invention also relates to an aircraft maintenance assistance system, comprising: display selection means; and a portable apparatus comprising a camera and augmented reality display means. According to the invention, the system has the following characteristics: the camera is connected to the display selection means, and arranged to acquire images and to send these images to the display selection means; the display selection means are connected to the camera and to the augmented reality display means, and are arranged to identify equipment of the aircraft present on said images, to determine an identifier of the identified equipment, said identifier useful, select maintenance assistance data relating to the identified equipment from the useful identifier, and send these data to the display means augmented reality; and the augmented reality display means is arranged to receive this maintenance aid data and, in response, to display in augmented reality images corresponding to this maintenance aid data. Preferably, the display selection means comprise: shape recognition means, arranged to identify the equipment of the aircraft from the images acquired by the camera, and outputting the useful identifier; and a display selection module, receiving as input the useful identifier, and, in response, providing the display means actually augmented with the maintenance assistance data relating to the identified equipment. BRIEF DESCRIPTION OF THE DRAWINGS The present invention will be better understood on reading the description of exemplary embodiments given purely by way of indication and in no way limiting, with reference to the appended drawings in which: FIG. 1 schematically illustrates a first embodiment of FIG. realization of a maintenance assistance system according to the invention; FIGS. 2A and 2B schematically illustrate a first embodiment of a maintenance assistance method according to the invention; FIGS. 3A and 3B schematically illustrate a second embodiment of a system and a method according to the invention; Figures 4A and 4B schematically illustrate a third embodiment of a system and method according to the invention; FIG. 5 schematically illustrates a detail of a fourth embodiment of a system according to the invention, associated with a fourth embodiment of a method according to the invention; FIG. 6 schematically illustrates a fifth embodiment of a system according to the invention, associated with a fifth embodiment of a method according to the invention; and Figure 7 schematically illustrates a detail of a sixth embodiment of a method according to the invention, associated with a sixth embodiment of a system according to the invention.
    [0006] DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS FIG. 1 schematically illustrates a first embodiment of a system 100 according to the invention for assisting the maintenance of an aircraft. The system 100 according to the invention comprises a portable device 120, and display selection means 130.
    [0007] The portable apparatus 120 and the display selection means 130 are in communication with each other, for example, a non-wired communication. The portable apparatus 120 includes a camera 121 and augmented reality display means 122. The augmented reality display means 122 designates any device, such as an observer observes a view of an environment surrounding it, on which are superimposed graphic data relating to this environment. These graphics data may include text, symbols, or any other visual, and may be referred to simply as "images." It can be a transparency overlay.
    [0008] In particular, the augmented reality display means 122 may designate: a device comprising a projector, projecting these graphic data directly onto the environment surrounding it; a device comprising a micro-screen and means for projecting to the eye of a user (for example, augmented reality vision glasses), so that the user equipped with this device and observing an environment visualizes at the same time this environment and an image from the micro-screen; or a device comprising display means, displaying an image of the surrounding environment on which said graphic data are superimposed (computer equipped with a screen and a camera, for example a laptop, a tablet digital, a smart phone, etc.). The user here designates a human operator, in particular a technician.
    [0009] The device 120 is portable because it has a small footprint, for example less than 0.05 m3, and a reduced weight, for example less than 5 kg. It can be worn by hand, or embedded on the user in the form for example glasses. The display selection means 130 designate calculation means comprising in particular a processor, for example a microprocessor. The system 100 according to the invention can be formed for example by: a computer-type device, in particular a laptop, a digital tablet, a smart phone, comprising a miniature camera, display means (display means, augmented reality 122) and a processor (display selection means 130); a computer-type device, notably a laptop, a digital tablet, a smart phone, comprising a miniature camera, a projector (augmented reality display means 122) and a processor (display selection means 130); or - glasses or an augmented reality vision helmet (augmented reality display means 122) provided with a camera, and connected by a wired or non-wired link to a processor (selection means 130).
    [0010] The camera 121 is adapted to acquire at least one image 1m of an external environment located especially inside an aircraft (not shown). In particular, the camera 121 is adapted to acquire images of an equipment of the aircraft.
    [0011] The display selection means 130 may also be portable. They can be formed in one piece with the portable device, or be deported and in particular wired communication with the portable device. The display selection means 130 as shown in FIG. 1 comprise form recognition means 131, and a display selection module 132. The camera 121 is in communication with the display selection means 130, so that to transmit them said acquired images. Within the display selection means, these images are processed more particularly by pattern recognition means 131.
    [0012] The display selection means 130 are adapted to identify, on at least one image acquired by the camera 121, a piece of equipment of the aircraft, and to determine an identifier of this equipment. In the example shown in FIG. 1, it is in particular the shape recognition means 131 that are adapted to carry out such an identification.
    [0013] For this, the shape recognition means 131 comprise a database, including geometric data including a plurality of equipment that may be present in the aircraft. In the database, each device is associated with an identifier. The shape recognition means therefore receives at least one image acquired by the camera 121, and outputs an identifier Id of an equipment present on the image, said useful identifier. This identification is carried out in real time. From the identifier Id, the display selection means 130 select maintenance assistance data relating to the identified equipment. In particular, the identifier Id is transmitted to a display selection module 132 which, in response, provides said maintenance assistance data. The display selection module 132 comprises, for example, a database in which each equipment identifier is associated with maintenance assistance data relating to this equipment. The equipment maintenance data relating to a device advantageously comprises details of the equipment itself, in particular its identification (for example its FIN identifier, for "Functional Item Number"). They may also include other data such as: instructions for handling the equipment; a list of elementary steps to be implemented when a maintenance operation is to be performed on this equipment, for example elementary steps to implement when you want to replace this equipment, move it, test its proper operation, etc. ; explanatory animations in three dimensions, a view of other equipment located behind, to prevent a user from dismantling equipment such as a panel to locate other equipment not directly visible, etc.
    [0014] The display selection means 130 are in communication with the augmented reality display means 122 for transmitting these maintenance assistance data. In particular, the display selection module 132 is in communication with the augmented reality display means 122, for transmitting these maintenance assistance data. From this maintenance aid data, the augmented reality display means 122 display, in augmented reality, images C corresponding to these maintenance assistance data.
    [0015] In particular, the augmented reality display means display the images corresponding to these maintenance assistance data so that a user observes a view of the identified equipment on which said images are superimposed. The augmented reality display can take various forms, as specified above: said images can be projected directly onto the equipment itself, by a projector; said images can be displayed on a micro-screen of augmented reality vision glasses, and projected into the user's field of vision so that this user can see the equipment with said images displayed in transparency on the view of the user. equipment; said images can be integrated into an image of the equipment displayed by a screen such as a screen of a digital tablet. The images corresponding to the maintenance aid data may include text, three-dimensional animation, a diagram, and so on. The augmented reality display preferably comprises the implementation of a tracking of the identified equipment, generally called "tracking". Thus, an image corresponding to the maintenance assistance data remains superimposed on a view of the identified equipment, regardless of the movements of the camera.
    [0016] The system 100, especially the portable apparatus 120, may include a man-machine interface (not shown). A user can formulate and send, to the display selection module 132, a query relating to the display of one of several images corresponding to maintenance assistance data. For example, the user may make a request to display information to help replace the equipment, or to repair a particular piece of equipment, etc. The maintenance assistance data displayed may relate to a maintenance operation to be performed, or relating to all the maintenance operations that can be performed on the equipment. FIGS. 2A and 2B schematically illustrate a first embodiment of a method, implemented in particular in the system 100 shown in FIG.
    [0017] Figure 2B illustrates the method according to the invention in the form of a flowchart. Figure 2A links each step of the flowchart to the elements of the system 100. Figure 2A corresponds to Figure 1 (in dashed lines), on which are added the elements of the flowchart shown in Figure 2B. The method according to the invention comprises the following steps: the camera 121 records the image lm of a device of the aircraft, and sends this image to the display selection means 130 (step 201); the display selection means 130 identify the equipment present on the image lm, and determine the identifier Id of this equipment, said useful identifier (step 202); from the identifier Id, the display selection means 130 selects the maintenance assistance data relating to the identified equipment, and sends these data to the augmented reality display means 122 (step 203); and the augmented reality display means 122 receive these data, and display in response, in augmented reality, the images corresponding to these maintenance assistance data.
    [0018] A second embodiment of a system and a method according to the invention will now be described with reference to FIGS. 3A and 3B. The system 300 will only be described for its differences with respect to the system 100 illustrated in FIG. 1. It is the same for the method. The reference numbers of FIG. 3A correspond to the reference numerals of FIG. 1, the number of the hundreds being replaced by a 3. The reference numerals 301, 302, 304 of FIG. 3B correspond to the reference numerals 101, 102, 104 of FIG. 2B.
    [0019] In the system 300 according to the invention, the display selection means 330 are arranged to receive a list Li of at least one identifier of interest, each identifier of interest being associated with a device of interest (step 3001 ). The display selection means 330 comprise a memory, for storing the list Li, and means of communication with an external device providing the list Li. The external device may consist of a human machine interface from which a user formulates the Li list, or a maintenance center having previously established the list based eg on fault messages transmitted by the aircraft. The equipment of interest in the list corresponds to a list of equipment to be controlled, handled, replaced, or repaired. This list is received in particular by the display selection module 332. The display selection module 332 is arranged to compare the useful identifier Id, and at least one identifier of interest of the list Li (step 3002) . Preferably, the display selection module 332 is arranged to compare the useful identifier Id, and each of the identifiers of interest. As a variant, the display selection module may receive data enabling it to preselect a sub-list of identifiers of interest that may correspond to the useful identifier, for example as a function of geolocation data.
    [0020] The display selection module 332 selects and supplies maintenance assistance data, only when the useful identifier is identical to one of the identifiers of interest in the list (step 3003). Thus, this second embodiment of the invention makes it possible to display maintenance assistance data only for a reduced number of equipment, corresponding to the only equipment to be handled, inspected, repaired or replaced. This limits a quantity of information displayed. The effectiveness of the assistance provided for the maintenance of the aircraft is thus improved.
    [0021] A third embodiment of a system and a method according to the invention will now be described with reference to FIGS. 4A and 4B. The system 400 will only be described for its differences with respect to the system 100 as illustrated in FIG. 1. The numerical references of FIG. 4A correspond to the reference numerals of FIG. 1, the number of the hundreds being replaced by a 4. The reference 401 of Figure 4B corresponds to the reference numeral 101 of Figure 2B. In the system 400 according to the invention, the display selection means 430 are arranged to receive a request Re, relating to at least one maintenance operation to be performed (step 4011). For this, the display selection means 430 comprise a memory, for storing the request Re, and means of communication with an external device providing the request Re. The external device may consist of a human machine interface from which a user formulates the request Re, or a maintenance center having previously established the request based eg of failure messages transmitted by the aircraft. The request Re corresponds to a list of maintenance operations formed for example of a series of computer files, each relating to a maintenance operation.
    [0022] The request Re contains various information relating to each maintenance operation to which it relates. This information comprises for example a list of at least one identifier of interest, each identifier of interest corresponding to a device to be manipulated, replaced, repaired for the implementation of at least one of the maintenance operations associated with the This information can also include a list of the positions of each of these devices. This information may also include maintenance assistance data relating to each of these devices.
    [0023] This request Re is received in particular by an extraction module 433. The extraction module 433 is arranged to analyze the request Re, and to extract at least one of: the list of identifiers of interest L, d, each identifier of interest corresponding to a device concerned by at least one maintenance operation associated with the request Re, said equipment of interest (step 4012); the set E of the maintenance assistance data relating to each of these devices, the data being classified in the set E according to the equipment concerned (step 4013); the list Lpc ,, of the positions of each of these equipments (step 4014).
    [0024] Preferably, each position of the list 40, simply corresponds to the indication of a zone of the aircraft in which the equipment is located (for example avionics bay, hold, etc.). In FIGS. 4A and 4B, there is shown the example in which the extraction module provides these three pieces of information, but it will be possible to implement variants comprising the extraction of only one or two of these three pieces of information. Steps 4012, 4013 and 4014 are advantageously performed in parallel.
    [0025] The portable device 420 comprises a geolocation module 423, adapted to calculate a position of the portable device, in particular a position relative to the aircraft. The geolocation module 423 provides this position to the display selection means (step 4001), in particular to the display selection module 432 and the shape recognition means 431. The position of the portable apparatus may correspond to a position relative to a landmark, for example a landmark in the aircraft. According to one preferred variant, the position of the portable device corresponds simply to the indication of an area of the aircraft in which the portable device is located (for example avionics bay, hold, etc.). It is advantageously the same areas as those used to define the positions of the equipment of interest. The geolocation module may include means for reading a marker such as an RFID chip, a bar code, a QR code, etc. Such markers are distributed in the different zones of the aircraft, thus making it possible to determine the area of the aircraft in which the portable device is located by simply reading one of these markers. Another example of a geolocation module will be described in detail with reference to FIG. 5. The shape recognition means 431 are arranged to identify the equipment present on an image lm provided by the camera, taking into account the position of the portable apparatus so as to select a restricted list of equipment that can match the equipment on said image (step 4021). For example, the database stored by the shape recognition means 431 further associates each equipment with a position in the aircraft. In particular, each equipment is associated with a zone of the aircraft as defined above. Thus, the shape recognition means can determine a restricted list of equipment located in the same area as the portable device, and then implement a pattern recognition. The calculations implemented are therefore greatly simplified. The information L, d, 40, and E are supplied to the display selection module 432. If the display selection module 432 receives the set E as defined above, it is not necessarily necessary that this module comprises a database in which each equipment identifier is associated with maintenance assistance data. The display selection module 332 is arranged to compare the useful identifier Id, and at least one identifier of interest of the list Lid (step 4002).
    [0026] Advantageously, the display selection module 432 is also arranged to compare the position of the portable device, and a position of a device of interest, provided by the list Lpc ,, (step 4022). This comparison preferably compares positions each corresponding to a zone of the aircraft. The result of this comparison then indicates whether the portable device and the equipment are or are not in the same area of the aircraft. The result of this comparison can be used by the display selection module in different ways, as detailed below. Preferably, step 4022 includes comparing the position of the portable apparatus, and each of the positions of a device of interest as provided by list 405.
    [0027] The display selection module 332 selects data, and provides them to the augmented reality display means (step 4033). In particular, three cases can be provided: equipment is present on the image provided by the camera, identified, and corresponds to one of the equipment of interest associated with the list Lid.
    [0028] In this case, the display selection means selects and sends maintenance assistance data relating to this equipment (see step 3003, FIG. 3B). Where appropriate, these data come from the set E as defined above. no equipment corresponding to one of the equipment of interest associated with the list Lid is identified from the image supplied by the camera, and step 4022 indicates that a device of interest is however close to the portable device. In this case, the display selection means selects and sends warning data, to warn a user that he is in the vicinity of this equipment. no equipment corresponding to one of the equipment of interest associated with the list Lid is identified from the image supplied by the camera, and step 4022 indicates that no equipment of interest is close of the portable device. In this case, the display selection means selects and sends guidance data, to guide a user to a device of interest, for example the nearest, or the next to be manipulated according to a recommended order of handling equipment affected by the maintenance operations. For the implementation of this guidance, the display selection module 332 uses a plane of the aircraft, including a three-dimensional modeling of the aircraft, stored in a memory of the display selection means. Then, in a step 4044, the augmented reality display means 422 display in augmented reality, as the case may be: - images corresponding to the maintenance assistance data received (see step 204, FIG. 2B); images corresponding to the received warning data, for example a flashing symbol, to warn the user that he is in the vicinity of a device of interest; images corresponding to the guidance data received, for example a symbol such as an arrow, for guiding a user towards a device of interest, for example the closest.
    [0029] The display selection means may implement a classification of the identifiers of interest, according to the respective positions of the corresponding equipment of interest and a plane of the aircraft. This classification is adapted to minimize the length of the user's journey in the aircraft when it is guided from one equipment of interest to another. The display selection means can implement a classification of the identifiers of interest according to the corresponding maintenance operation, so that a user does not begin a new maintenance operation until he has completed a maintenance operation. previously started. This embodiment of the invention makes it possible to offer a user a display adapted to maintenance operations to be performed, and to its position. Maintenance is thus further facilitated and its duration reduced. This embodiment is particularly complete, and it will be easy to envisage numerous variants in which only a portion of the steps and modules described are implemented, for example the extraction and use of only certain data from a request. relating to at least one maintenance operation to be performed.
    [0030] FIG. 5 illustrates a particular example of geolocation module 523, as mentioned with reference to FIG. 4A. FIG. 5 also illustrates sub-steps of a step 4001 as mentioned with reference to FIG. 4B, during which the portable apparatus 420 provides its position to the display selection means 430.
    [0031] The geolocation module 523 comprises: - 5231 geolocation means, providing a position of the portable device relative to a fixed landmark, called the raw position. These means comprise for example a GPS satellite navigation device. These means correspond to a substep of determining said raw position; a memory 5232, receiving a three-dimensional modeling of the aircraft. This memory corresponds to a substep of comparison between the raw position and the modeling; and comparison means 5233 receiving as input the raw position and the modeling, and outputting the position P of the portable apparatus relative to the aircraft. These means correspond to a substep of determining the position of the portable device.
    [0032] FIG. 6 schematically illustrates a fifth embodiment of a system 600 according to the invention, associated with a fifth embodiment of a method according to the invention. The system 600 will only be described for its differences with respect to the system 100 as illustrated in FIG. 1. The reference numerals of FIG. 6 correspond to the reference numerals of FIG. 1, the number of the hundreds being replaced by a numeral 6. According to this embodiment, the display selection means 630 comprise communication means (not shown) with a maintenance center 640. The maintenance center sends the display selection means 630 a request Re as defined with respect to the Figures 4A and 4B. The communication may be a non-wired communication (eg wifi). In a variant, the communication is implemented by a direct contact between a contact point of the display selection means 630 and a loading terminal of the request, located in the maintenance center. This request may be accompanied by guidance data to the aircraft concerned by the maintenance operations to be performed. Thus, the augmented reality display means can display, in augmented reality, images corresponding to these guidance data to bring the user to the aircraft.
    [0033] The method implemented in the system 600 comprises sending, by the maintenance center 640 and to the display selection means 630, at least one request Re. The request Re is then stored in a memory 634. display selection means 630. The memory 634 makes it possible to store a list of several requests Re, for example so that a user then carries out the maintenance of several aircraft. The memory 634 transmits a first request to the extraction means 633. The extraction means 633 notably provide the list L, d, as described with reference to FIGS. 4A and 4B. The display selection means 630 also comprise means 635 for formulating a request Re. The means 635 comprise in particular a human-machine interface, receiving Din input data supplied by a user, and formulating the Re request from these Din input data. The method implemented in the system 600 may then comprise a step during which the user sends Din data to the means 635, by means of a human-machine interface of said means 635. For example, the user consults the log of board (or "logbook") of an aircraft. This log summarizes the incidents that occurred previously in the aircraft. The user deduces maintenance operations to be performed. He then drives the formulation of the request Re, and is then assisted in the implementation of said operations. Alternatively, the user carries out the maintenance operations corresponding to a request provided by the maintenance center, and notes that a fault persists. It can then deduce a new maintenance operation to be performed, and drive accordingly the formulation of a new request Re.
    [0034] FIG. 7 schematically illustrates a detail of a sixth embodiment of a method according to the invention, associated with a sixth embodiment of a system according to the invention. This sixth embodiment is an improvement of the embodiment of FIG. 4B.
    [0035] Numerals 7011 and 7001 correspond to the reference numerals 4011 and 4001 of FIG. 4B. The Re request received in step 7011 corresponds to a list of several maintenance operations, each associated with a position of interest. Said position of interest may correspond in particular to an area of the aircraft in which are all the equipment relating to the maintenance operation. In a step 7015, the display selection means extract from the request Re the positions of interest associated with each maintenance operation, and classify the maintenance operations according to their position of interest and from a modeling of the aircraft. This classification aims for example to minimize a user's journey in the aircraft. The corresponding system then comprises an extraction module and a classification module belonging to the display selection means, and adapted to implement step 7015. In a step 7023, the display selection means compare the position of the portable apparatus received at step 7001, and the positions of interest considered in the order of the classification. The display selection means selects and sends guidance data, for guiding a user from one area of the aircraft to the next, so that the user implements the maintenance operations in the order of classification. At step 7044, the augmented reality display means receives these data. In response, they display in augmented reality images corresponding to these guidance data. These steps can for example be integrated into a method as described with reference to FIG. 4B, so as to define the following three cases: the portable device is not at the position of interest corresponding to the next operation of maintenance to be performed, according to the order of the classification: the display selection means selects and sends guidance data to this position of interest; the portable device is in said position of interest, and no equipment associated with the corresponding maintenance operation is identified: the display selection means selects and sends warning data informing that such equipment is near; the portable apparatus is in said position of interest, and equipment associated with the corresponding maintenance operation is identified: the display selection means selects and sends maintenance assistance data relating to that equipment. The invention is not limited to the examples described. In addition, it will be possible to easily make many variants implementing only part of the elements described, as long as the essential elements of the invention are implemented.
    权利要求:
    Claims (15)
    [0001]
    REVENDICATIONS1. A method of assisting the maintenance of an aircraft, characterized in that it is implemented in a system (100; 300; 400; 600) comprising: display selection means (130; 330; 430; 630); and a portable apparatus (120; 320; 420; 620) comprising a camera (121; 321; 421; 621) and augmented reality display means (122; 322; 422; 622); and in that the method comprises the following steps: the camera (121; 321; 421; 621) acquires images (1m) of equipment from the aircraft and sends these images to the display selection means (130); 330, 430, 630); the display selection means (130; 330; 430; 630) identify the equipment present on said images and determine an identifier (Id) of the identified equipment, said useful identifier; from this useful identifier (Id), the display selection means (130; 330; 430; 630) select maintenance assistance data relating to the identified equipment, and send this data to the means of augmented reality display; In response, the augmented reality display means (122; 322; 422; 622) displays, in fact augmented, images (C D) corresponding to the maintenance aid data.
    [0002]
    2. Method according to claim 1, characterized in that the display selection means (330; 430; 630) implement the following steps: reception of a list (Li; Lid) of at least one identifier of interest, each identifier of interest corresponding to a device of interest; comparison between the useful identifier (Id) and at least one identifier of interest of the list; andselecting and sending said maintenance aid data only when the useful identifier (Id) corresponds to an identifier of interest in the list (Li; Lid).
    [0003]
    3. Method according to claim 2, characterized in that the display selection means (430; 630) implement the following steps: receiving a request (Re) relating to at least one maintenance operation to be performed; extracting, from this request (Re), the list (Ltd) of at least one identifier of interest, an identifier of interest corresponding to a device of interest, and the equipment of interest forming a piece of equipment associated with one of the at least one maintenance operation to be performed.
    [0004]
    4. Method according to claim 3, characterized in that the display selection means (430) implement an extraction, for each identifier of interest from the list, and from the request (Re) relating to the least one maintenance operation to be performed, a position of the corresponding equipment of interest.
    [0005]
    5. Method according to claim 3 or 4, characterized in that the display selection means (430) implement an extraction, for each identifier of interest from the list, and from the request (Re) relative at least one maintenance operation to be performed, corresponding maintenance assistance data.
    [0006]
    The method of any one of claims 1 to 5, characterized in that the portable apparatus (420) provides the display selection means (430) with a position of the portable apparatus.
    [0007]
    7. The method of claim 6, characterized in that the portable device (420) implements the following steps: determining a raw position of the portable device relative to a reference point; comparison between this raw position and a modeling of the aircraft; determining the position (P) of the portable apparatus relative to said aircraft.
    [0008]
    The method according to claim 6 or 7, characterized in that to identify the equipment present on said images, the display selection means (430) selects a list of equipment located near the portable device, from the position (P) of the portable apparatus and the positions of the equipment of the aircraft.
    [0009]
    9. Method according to any one of claims 6 to 8, characterized in that the display selection means (430) implement a comparison between the position (P) of the portable device and a position of a equipment of interest, the equipment of interest corresponding to an identifier of interest of a list (Li, L, d) of identifiers of interest such that the display selection means do not send the data of only when the useful identifier (Id) corresponds to one of the identifiers of interest in the list.
    [0010]
    The method of claim 9, characterized in that: the display selection means (430) selects and sends to the augmented reality display means (422) guidance data to a device of interest; and the augmented reality display means displays, in augmented reality, images (C) corresponding to these guidance data.
    [0011]
    Method according to claim 9 or 10, characterized in that: the display selection means (430) selects and sends to the augmented reality display means (422) warning data relating to the proximity of the display. a device of interest, when the position (P) of the portable apparatus corresponds to the position of a device of interest; and the augmented reality display means (422) display, in augmented reality, images corresponding to these alert data.
    [0012]
    12. Method according to any one of claims 3 to 11, characterized in that it comprises a step of formulation, by a human operator and with the aid of a human-machine interface (635), of the request ( Re) relating to at least one maintenance operation to be performed.
    [0013]
    13. Method according to any one of claims 3 to 12, characterized in that it comprises a step of sending, by a maintenance center (640), the request (Re) relating to at least one maintenance operation. to achieve.
    [0014]
    14. Aircraft Maintenance Support System (100; 300; 400; 600), comprising: display selection means (130; 330; 430; 630); and a portable apparatus (120; 320; 420; 620) comprising a camera (121; 321; 421; 621) and augmented reality display means (122; 322; 422; 622); characterized in that: the camera (121; 321; 421; 621) is connected to the display selection means (130; 330; 430; 630), and arranged to acquire images (lm) and to send these images to display selection means (130; 330; 430; 630); the display selection means (130; 330; 430; 630) are connected to the camera (121; 321; 421; 621) and to the display means; augmented reality display (122; 322; 422; 622), and are arranged to identify equipment of the aircraft present on said images, determine an identifier (Id) of the identified equipment, said useful identifier, select data maintenance assistance relating to the equipment identified from the useful identifier, and send this data to the augmented reality display means (122; 322; 422; 622); and the augmented reality display means (122; 322; 422; 622) are arranged to receive this maintenance aid data and, in response, to display in augmented reality images (D) corresponding to these data of help with maintenance.
    [0015]
    15. System (100; 300; 400; 600) according to claim 14, characterized in that the display selection means (130; 330; 430; 630) comprise: shape recognition means (131; 331; 431; 631), arranged to identify the equipment of the aircraft from the images (lm) acquired by the camera, and outputting the useful identifier (Id); and a display selection module (132; 332; 432; 632), inputting the useful identifier (Id), and responsively providing the augmented display means (122; 322; 422; 622) maintenance assistance data for the identified equipment.
    类似技术:
    公开号 | 公开日 | 专利标题
    FR3023948A1|2016-01-22|METHOD FOR AIDING THE MAINTENANCE OF AN AIRCRAFT BY INCREASED REALITY
    JP6859505B2|2021-04-14|Image-based vehicle damage determination methods, devices and electronic devices
    KR20190069457A|2019-06-19|IMAGE BASED VEHICLES LOSS EVALUATION METHOD, DEVICE AND SYSTEM,
    EP3520044A1|2019-08-07|Picture-based vehicle loss assessment method and apparatus, and electronic device
    CA2830916A1|2012-09-27|Augmented reality system for product selection
    CN104977717A|2015-10-14|Head mounted display presentation adjustment
    EP3719703A1|2020-10-07|Vehicle damage identification processing method, processing device, client and server
    CN109410218B|2020-08-11|Method and apparatus for generating vehicle damage information
    US20160259928A1|2016-09-08|Real-time presence verification
    EP2874106A1|2015-05-20|System and method for aircraft failure diagnosis
    US20160189170A1|2016-06-30|Recognizing Customers Requiring Assistance
    US20170084015A1|2017-03-23|Examining defects
    KR102094341B1|2020-03-27|System for analyzing pot hole data of road pavement using AI and for the same
    Temel et al.2018|Traffic signs in the wild: Highlights from the ieee video and image processing cup 2017 student competition [sp competitions]
    WO2017165332A1|2017-09-28|2d video analysis for 3d modeling
    US20210103973A1|2021-04-08|Generating a product recommendation based on a user reaction
    EP3404513A1|2018-11-21|Information processing apparatus, method, and program
    Stanimirovic et al.2014|[Poster] A Mobile Augmented reality system to assist auto mechanics
    KR20160106649A|2016-09-12|Evaluation of augmented reality skins
    JP2017162103A|2017-09-14|Inspection work support system, inspection work support method, and inspection work support program
    FR2997493A1|2014-05-02|Method for updating data set of navigation database for navigation of vehicles, involves recording feedback messages regarding data set, and adapting attribute in data set if preset number of feedback messages indicates error of attribute
    WO2018087462A1|2018-05-17|Individual visual immersion device for a moving person with management of obstacles
    EP2939902B1|2018-12-26|Method and device for determining a railway itinerary
    Nassar et al.2019|Assessing data quality of annotations with Krippendorff alpha for applications in computer vision
    CN107945126B|2022-02-18|Method, device and medium for eliminating spectacle frame in image
    同族专利:
    公开号 | 公开日
    US10049111B2|2018-08-14|
    US20160019212A1|2016-01-21|
    FR3023948B1|2017-12-22|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20140022281A1|2012-07-18|2014-01-23|The Boeing Company|Projecting airplane location specific maintenance history using optical reference points|
    US9074899B2|2009-05-11|2015-07-07|Acer Incorporated|Object guiding method, mobile viewing system and augmented reality system|
    CA2895778C|2012-12-20|2021-10-19|Accenture Global Services Limited|Context based augmented reality|
    US9019128B1|2013-05-21|2015-04-28|The Boeing Company|Augmented reality aircraft management system|
    US9446856B1|2013-07-31|2016-09-20|Rockwell Collins, Inc.|Method and apparatus for maintaining aircraft|ITUB20154627A1|2015-10-13|2017-04-13|Sematic S P A|PROCEDURE FOR MAINTENANCE OF AN ELECTROMECHANICAL DEVICE|
    JP6432551B2|2016-03-09|2018-12-05|横河電機株式会社|Equipment maintenance device, equipment maintenance system, equipment maintenance method, equipment maintenance program, and recording medium|
    ES2643863B1|2016-05-24|2018-10-26|Sonovisión Ingenieros España, S.A.U.|METHOD FOR PROVIDING BY GUIDED INCREASED REALITY, INSPECTION AND SUPPORT IN INSTALLATION OR MAINTENANCE OF PROCESSES FOR COMPLEX ASSEMBLIES COMPATIBLE WITH S1000D AND DEVICE THAT MAKES SAME USE|
    FR3056966B1|2016-10-04|2018-10-05|Safran Electrical & Power|METHOD OF LOCATING DEFECT IN ELECTRICAL HARNESSES|
    BE1025077B1|2017-03-22|2018-10-23|Safran Aero Boosters S.A.|TURBOMACHINE COVER WITH SCREEN|
    FR3071080B1|2017-09-12|2019-09-06|Safran Landing Systems|METHOD FOR INSPECTING EQUIPMENT OR AIRCRAFT PARTS|
    US11062517B2|2017-09-27|2021-07-13|Fisher-Rosemount Systems, Inc.|Virtual access to a limited-access object|
    US10760815B2|2017-12-19|2020-09-01|Honeywell International Inc.|Building system commissioning using mixed reality|
    US10529104B2|2017-12-19|2020-01-07|GM Global Technology Operations LLC|Virtual vehicle skin|
    US10410426B2|2017-12-19|2019-09-10|GM Global Technology Operations LLC|Augmented reality vehicle user interface|
    US10706628B2|2018-02-28|2020-07-07|LenovoPte. Ltd.|Content transfer|
    IT201800005543A1|2018-05-21|2019-11-21|PROCEDURE TO ASSIST AN OPERATOR IN THE PERFORMANCE OF INTERVENTIONS ON A OPERATING MACHINE USING AUGMENTED REALITY TECHNIQUES.|
    WO2019224857A1|2018-05-21|2019-11-28|I.M.A. Industria Macchine Automatiche S.P.A.|Method to assist an operator in performing interventions on an operating machine|
    FR3084173A1|2018-07-18|2020-01-24|Holomake|MOTORIZED MECHANICAL SERVO SYSTEM OF A HOLOGRAPHIC PLAN FOR MANUAL PRECISION GUIDANCE|
    IT201800007987A1|2018-08-09|2020-02-09|Ima Industria Macch Automatiche Spa|PROCEDURE FOR ASSISTING AN OPERATOR IN PERFORMING INTERVENTIONS ON A OPERATING MACHINE|
    JP6692867B2|2018-09-07|2020-05-13|日本航空株式会社|System and method for identifying the interior of a structure|
    JP6651189B1|2019-03-29|2020-02-19|株式会社 情報システムエンジニアリング|Data structure, learning method and information providing system for machine learning|
    US11182969B2|2019-10-29|2021-11-23|Embraer S.A.|Spatial localization using augmented reality|
    US11237534B2|2020-02-11|2022-02-01|Honeywell International Inc.|Managing certificates in a building management system|
    法律状态:
    2015-06-26| PLFP| Fee payment|Year of fee payment: 2 |
    2016-01-22| PLSC| Publication of the preliminary search report|Effective date: 20160122 |
    2016-07-21| PLFP| Fee payment|Year of fee payment: 3 |
    2017-07-24| PLFP| Fee payment|Year of fee payment: 4 |
    2018-07-25| PLFP| Fee payment|Year of fee payment: 5 |
    2020-07-21| PLFP| Fee payment|Year of fee payment: 7 |
    2021-07-27| PLFP| Fee payment|Year of fee payment: 8 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1457015A|FR3023948B1|2014-07-21|2014-07-21|METHOD FOR AIDING THE MAINTENANCE OF AN AIRCRAFT BY INCREASED REALITY|FR1457015A| FR3023948B1|2014-07-21|2014-07-21|METHOD FOR AIDING THE MAINTENANCE OF AN AIRCRAFT BY INCREASED REALITY|
    US14/805,115| US10049111B2|2014-07-21|2015-07-21|Maintenance assistance for an aircraft by augmented reality|
    [返回顶部]